Visualizing Uncertainty in Predictive Models

In many scientific fields, models are used to characterize relationships and processes, as well as to predict outcomes from initial conditions and inputs. These models can support the decision-making process by allowing investigators to consider the likely effects of possible interventions and identify efficient ways to achieve desired outcomes. Machine learning research on constructing complex models (such as Bayesian networks) typically focuses on maximizing predictive accuracy, or other measures of model quality. Model confidence refers to the estimated certainty in the classifications produced by the model. We describe a new framework for improving the understanding of complex models by drawing upon the strengths of both machine learning and data visualization. These two disciplines complement each other to combine the benefits of intelligent automatic support for design and analysis with visual representations and interactions that boost human abilities. We leverage these approaches to address the challenges of developing, understanding, and using complex models to facilitate scientific discovery and informed decision-making. Our focus is on understanding the uncertainty that is associated with model predictions. This uncertainty arises from several sources. Sample uncertainty occurs when regions of the instance space are not well represented in the training data, and predictions are therefore based on sparse information. Model instability occurs when model predictions vary, depending on the training data that was used to construct the model. Prediction variability occurs when a given observation may have noisy at-